Support pprof profiling feature #13502

JophieQu · 2025-09-18T07:18:39Z

This PR introduces a new pprof profiling analysis feature for Go services in SkyWalking.
It enables on-demand performance profiling of Go applications through SkyWalking’s UI, including CPU, Heap, Block, Mutex, Goroutine, ThreadCreate, and Allocs events.

If this is non-trivial feature, paste the links/URLs to the design doc.
Update the documentation to include this new feature.
Tests(including UT, IT, E2E) are added to verify the new feature.
If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the CHANGES log.

oap-server/server-starter/src/main/resources/bydb.yml

oap-server/server-starter/src/main/resources/application.yml

…nto feat/pprof

mrproliu · 2025-10-22T11:18:01Z

.../apache/skywalking/oap/server/storage/plugin/elasticsearch/query/PprofTaskLogQueryEsDAO.java

+    }
+}
+
+


Please remove empty lines.

wu-sheng · 2025-10-22T12:15:33Z

docs/en/setup/backend/backend-go-app-profiling.md

@@ -0,0 +1,107 @@
+# Go App Profiling


This file should be linked from doc menu.

wu-sheng · 2025-10-22T12:16:37Z

docs/en/setup/backend/backend-go-app-profiling.md

+
+Go App Profiling uses the Pprof for sampling
+
+Pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).


Suggested change

Pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).

pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).

pprof is a project name. They don't use capital first letter, ref to https://github.com/google/pprof

wu-sheng · 2025-10-22T12:27:17Z

docs/en/setup/backend/backend-go-app-profiling.md

+Pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).
+
+It is delivered to the agent in the form of a task, allowing it to be enabled or disabled dynamically.
+When service encounters performance issues (cpu usage, memory allocation, etc.), Pprof task can be created.


Suggested change

When service encounters performance issues (cpu usage, memory allocation, etc.), Pprof task can be created.

When service encounters performance issues (CPU usage, memory allocation, etc.), pprof task can be created.

Please fix nits in your words.

docs/en/setup/backend/backend-go-app-profiling.md

wu-sheng · 2025-10-22T12:44:16Z

docs/en/changes/changes.md

 * Self Observability: add `metrics_aggregation_queue_used_percentage` and `metrics_persistent_collection_cached_size` metrics for the OAP server.
 * Optimize metrics aggregate/persistent worker: separate `OAL` and `MAL` workers and consume pools. The dataflow signal drives the new MAL consumer, 
  the following table shows the pool size，driven mode and queue size for each worker.
+* Support pprof profiling feature. 


Suggested change

* Support pprof profiling feature.

* Support the go agent(0.7.0 release) bundled pprof profiling feature.

…nto feat/pprof

wu-sheng

some nits.

wu-sheng · 2025-10-24T15:44:25Z

.../java/org/apache/skywalking/oap/server/network/trace/component/command/PprofTaskCommand.java

+     * pprof taskId
+     */
+    private String taskId;
+    // Type of profiling (CPU/Heap/Block/Mutex/Goroutine/Threadcreate/Allocs)


The comment format is not consistent, please fix this.

wu-sheng · 2025-10-24T15:46:08Z

docs/en/changes/changes.md

 * Self Observability: add `metrics_aggregation_queue_used_percentage` and `metrics_persistent_collection_cached_size` metrics for the OAP server.
 * Optimize metrics aggregate/persistent worker: separate `OAL` and `MAL` workers and consume pools. The dataflow signal drives the new MAL consumer, 
  the following table shows the pool size，driven mode and queue size for each worker.
+* Support the go agent(0.7.0 release) bundled pprof profiling feature. 


You injected the change logs into a wrong place move it to the last line of oap changes section.

wu-sheng · 2025-10-24T15:46:32Z

docs/en/concepts-and-designs/profiling.md


+### Go App Profiling
+
+Go App Profiling uses the [Pprof](https://github.com/google/pprof) for sampling.


Suggested change

Go App Profiling uses the [Pprof](https://github.com/google/pprof) for sampling.

Go App Profiling uses the [pprof](https://github.com/google/pprof) for sampling.

wu-sheng · 2025-10-24T15:48:03Z

docs/en/setup/backend/backend-go-app-profiling.md

+
+Go App Profiling uses the pprof for sampling
+
+pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).


Suggested change

pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).

pprof is bundled within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).

typo?

JophieQu added 2 commits September 18, 2025 11:11

feat: support pprof feature

41acb04

fix

6b4640f

wankai123 reviewed Sep 19, 2025

View reviewed changes

oap-server/server-starter/src/main/resources/bydb.yml Outdated Show resolved Hide resolved

wankai123 reviewed Sep 19, 2025

View reviewed changes

oap-server/server-starter/src/main/resources/application.yml Outdated Show resolved Hide resolved

JophieQu added 26 commits September 24, 2025 20:50

fix

a54d8dc

fix: rollback yaml

d069e68

fix

1afb571

fix

2578bd1

add jdbc & es storage

4c03a81

test ci

5520d01

update proto

c965482

fix query submules

c467076

fix ci

5000e71

fix ci

cce039d

ci test (to rollback)

3989494

fix

232cdc4

Merge remote-tracking branch 'origin/master' into ci

1dc8fcd

fix

e8f44ad

fix e2e

f46887f

fix e2e

f49e7a3

fix e2e docker path

3449d89

fix

26956b0

fix

baaf886

fix

62fd3fb

fix duration

88692d1

fix

843585f

fix port

07fe14c

fix e2e

2e62876

fix e2e

02a5b55

fix e2e

ed0403a

wu-sheng and others added 5 commits October 21, 2025 19:21

Merge branch 'master' into feat/pprof

36a83c9

fix pprof cache

5b6ffae

fix get pprof task

6bb29d4

Merge branch 'ci1' into feat/pprof

f979edd

Merge branch 'feat/pprof' of https://github.com/JophieQu/skywalking i…

8adfd28

…nto feat/pprof

mrproliu reviewed Oct 22, 2025

View reviewed changes

delete blank lines

f613cbd

mrproliu previously approved these changes Oct 22, 2025

View reviewed changes

wu-sheng reviewed Oct 22, 2025

View reviewed changes

docs/en/setup/backend/backend-go-app-profiling.md

@@ -0,0 +1,107 @@

# Go App Profiling

Copy link

Member

wu-sheng Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be linked from doc menu.

wu-sheng reviewed Oct 22, 2025

View reviewed changes

docs/en/setup/backend/backend-go-app-profiling.md Show resolved Hide resolved

wu-sheng reviewed Oct 22, 2025

View reviewed changes

fix doc & mock oap

02d2964

JophieQu dismissed mrproliu’s stale review via 02d2964 October 24, 2025 08:23

JophieQu and others added 3 commits October 24, 2025 16:27

roll back proto

4f76698

roll back

49d84a0

Merge branch 'master' into feat/pprof

79a74bb

mrproliu previously approved these changes Oct 24, 2025

View reviewed changes

JophieQu added 2 commits October 24, 2025 22:20

fix codestyle

ce5384b

Merge branch 'feat/pprof' of https://github.com/JophieQu/skywalking i…

5226098

…nto feat/pprof

JophieQu dismissed mrproliu’s stale review via 5226098 October 24, 2025 14:21

fix

4c47d09

mrproliu previously approved these changes Oct 24, 2025

View reviewed changes

wu-sheng reviewed Oct 24, 2025

View reviewed changes

fix doc

a895cba

JophieQu dismissed mrproliu’s stale review via a895cba October 24, 2025 16:10

fix doc

c2825e7

wu-sheng approved these changes Oct 25, 2025

View reviewed changes

wu-sheng merged commit eabddf2 into apache:master Oct 25, 2025
176 checks passed


		Go App Profiling uses the Pprof for sampling

		Pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).

	When service encounters performance issues (cpu usage, memory allocation, etc.), Pprof task can be created.
	When service encounters performance issues (CPU usage, memory allocation, etc.), pprof task can be created.

	* Support pprof profiling feature.
	* Support the go agent(0.7.0 release) bundled pprof profiling feature.


		### Go App Profiling

		Go App Profiling uses the [Pprof](https://github.com/google/pprof) for sampling.

	pprof is bound within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).
	pprof is bundled within the auto-instrument agent and corresponds to [In-Process Profiling](../../concepts-and-designs/profiling.md#in-process-profiling).

Support pprof profiling feature #13502

Support pprof profiling feature #13502

Uh oh!

Conversation

JophieQu commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wu-sheng left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JophieQu commented Sep 18, 2025 •

edited

Loading